Extracting meta tags attribute using wget [migrated]

Posted by Amit on Pro Webmasters See other posts from Pro Webmasters or by Amit
Published on 2011-06-26T09:35:32Z Indexed on 2011/06/26 16:31 UTC
Read the original article Hit count: 265

Filed under:

html

|

meta-keywords

I have a file having some URLs per line. I need to extract the "keywords" present in the tags i.e. if there is meta tag for "keywords" then i want to get "content" value for it. Example: if the web-page has this meta-tag then for that URL i want "wikipedia,encyclopedia" to be extracted.

One approach is to download the web-page using "wget" and then parse it using some standard HTML parser.

I was wondering is there any better way to do this without downloading the entire web-page.

© Pro Webmasters or respective owner

Related posts about html

Install usblib package - Ubuntu

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I need the package libusb for another package I am installing. I tried the following which seemed to install the package, sudo apt-get install libusb-dev but when I try to install the other package I get, configure: error: Package requirements (libusb-1.0 >= 0.9.1) were not met: No package… >>> More
Prevent malicious vulnerability scan increasing load on a server

as seen on Server Fault - Search for 'Server Fault'
Hi all, this week we have been suffering some malicious vulnerability scans to our servers, increasing the load on them, making them nearly unusable. The attack is easy to defend, just blocking the offending ip, but only after discovering it. Is there any form of prevent it? Is it normal that… >>> More
can't install psycopg2 in my env on mac os x lion

as seen on Server Fault - Search for 'Server Fault'
I tried install psycopg2 via pip in my virtual env, but got this error: ld: library not found for -lpq (full log here: http://pastebin.com/XdmGyJ4u ) I tried install postgres 9.1 from .dmg and via port, (gksks)iMac-Alexander:~ lorddaedra$ locate libpq /Developer/SDKs/MacOSX10.7.sdk/usr/include/libpq /Developer/SDKs/MacOSX10… >>> More
Bitnami redmine error SVN

as seen on Server Fault - Search for 'Server Fault'
I'm installing the Bitnami Redmine stack (redmine + subversion). Firstly I install configure and test it locally (Ubuntu 14.04 LTS). And everything is OK. I install Bitnami stack on server (Red Hat 4.4.7-4) and configure SVN. I commit files into SVN and connect project into Redmine with SVN repository… >>> More
Can the .htaccess file slow down a website to a crawl? If so, are there better ways to solve these problems with different rewrite rules and such?

as seen on Pro Webmasters - Search for 'Pro Webmasters'
here is my htaccess file...... RewriteCond %{REQUEST_URI} ^/patients/billing/FAQ_billing\.html$ [OR] RewriteCond %{REQUEST_URI} ^/patients/billing/getintouch\.html$ RewriteRule ^patients/billing/(.*)\.html$ $1.php [L,NC] RewriteCond %{REQUEST_URI} ^/patients/findadoctor/a\.html$ [OR] RewriteCond… >>> More

Related posts about meta-keywords

ASP.NET Meta Keywords and Description

as seen on Johnny Coder - Search for 'Johnny Coder'
Some of the ASP.NET 4 improvements around SEO are neat. The ASP.NET 4 Page.MetaKeywords and Page.MetaDescription properties, for example, are a welcomed change. There’s nothing earth-shattering going on here – you can now set these meta tags via your Master page’s code behind rather than… >>> More
Setting Meta tags for a website

as seen on Pro Webmasters - Search for 'Pro Webmasters'
I have made an Asp.net MVC website and am not well versed about SEO techniques, so I want a little guidance in setting the appropriate meta tags for the website. My website is dynamic and has two types of Pages: Category and Product There are two tables in the database for Category and Product.… >>> More
Référencement : Google ressuscite la balise « Meta Keywords » pour son service Google Actualités

as seen on Developper.com - Search for 'Developper.com'
Référencement : Google ressuscite la balise « Meta Keywords » Pour son service Google Actualités Détrompez-vous, les balises META keyword ne sont pas complètement tombées dans les oubliettes. Google annonce sur le site officiel de Google News une nouvelle balise-meta appelée « news_keywords… >>> More
SEO Meta Keywords - How to Learn SEO

as seen on Ezine Articles - Search for 'Ezine Articles'
Every Internet business and Internet marketer has Search Engine Results Pages (also known as SERP or SERPS) constantly on their minds - they are one of the most important things to think about. Read to get started learning some of the basics to drive that free traffic to your websites. >>> More
Dynamic meta description and keyword tags for your MasterPages

as seen on ASP.net Weblogs - Search for 'ASP.net Weblogs'
Today we're going to look at a technique for dynamically inserting meta tags into your master pages. By taking control of the head tag and inserting your own HtmlMeta you can easily customise these tags.Might have noticed that when you create a new master page in visual studio your <head>… >>> More